Training a machine learning system to detect an excursion of a cmp component using time-based sequence of images

ABSTRACT

Monitoring operations of a polishing system includes obtaining a time-based sequence of reference images of a component of the polishing system performing operations during a test operation of the polishing system, receiving from a camera a time-based sequence of monitoring images of an equivalent component of an equivalent polishing system performing operations during polishing of a substrate, determining a difference value for the time-based sequence of monitoring images by comparing the time-based sequence of reference images to the time-based sequence of monitoring image using an image processing algorithm, determining whether the difference value exceeds a threshold, and in response to determining the difference value exceeds the threshold, indicating an excursion.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a continuation of U.S. application Ser. No.17/678,939, filed Feb. 23, 2022, which claims priority to U.S.Provisional Application Ser. No. 63/157,616, filed on Mar. 5, 2021, thedisclosures of which are incorporated by reference.

TECHNICAL FIELD

The present disclosure generally relates to chemical mechanicalpolishing (CMP), and more particularly, to detecting an excursion of aCMP components using a time-based sequence of images (e.g., videoimages).

BACKGROUND

An integrated circuit is typically formed on a substrate (e.g. asemiconductor wafer) by the sequential deposition of conductive,semiconductive or insulative layers on a silicon wafer, and by thesubsequent processing of the layers.

One fabrication step involves depositing a filler layer over anon-planar surface and planarizing the filler layer. For certainapplications, the filler layer is planarized until the top surface of apatterned layer is exposed or a desired thickness remains over theunderlying layer. In addition, planarization may be used to planarizethe substrate surface, e.g., of a dielectric layer, for lithography.

Chemical mechanical polishing (CMP) is one accepted method ofplanarization. This planarization method typically requires that thesubstrate be mounted on a carrier head. The exposed surface of thesubstrate is placed against a rotating polishing pad. The carrier headprovides a controllable load on the substrate to push it against thepolishing pad. In some situations, the carrier head includes a membranethat forms multiple independently pressurizable radially concentricchambers, with the pressure in each chamber controlling the polishingrate in each corresponding region on the substrate. A polishing liquid,such as slurry with abrasive particles, is supplied to the surface ofthe polishing pad.

Image processing aims to process one or more image frames usingdifferent algorithms, including image compressing, image filtering,image storage, and image comparison. Image comparison can be devoted tonoise reduction, image matching, image encoding, and restoration, whichcan be implemented by one or more computers in one or more locationsusing one or more image comparison algorithms. Image comparisonalgorithms can determine a level of similarity or difference between oneor more images based on image characteristics, e.g., pixel valuesrepresenting brightness, color, and transparency, or metric distances(e.g., Hausdorff distance or other suitable distances) measuringrespective distances between each pair of components in an image oracross different image frames, or feature kernels that represent localimage patches and used for matching features between images. The imagecomparison algorithm is further assisted with any suitablepre-processing steps such as pixel intensity adjustment, normalization,or homomorphic filtering, to name just a few examples.

Video images can also be processed using machine learning algorithms.Neural networks are machine learning models that employ one or morelayers of nonlinear units to predict an output for a received input.Some neural networks include one or more hidden layers in addition to anoutput layer. The output of each hidden layer is used as input to thenext layer in the network, i.e., the next hidden layer or the outputlayer. Each layer of the network generates an output from a receivedinput in accordance with current values of a respective set ofparameters.

SUMMARY

In one aspect, monitoring operations of a polishing system includesobtaining a time-based sequence of reference images of a component ofthe polishing system performing operations during a test operation ofthe polishing system, receiving from a camera a time-based sequence ofmonitoring images of an equivalent component of an equivalent polishingsystem performing operations during polishing of a substrate,determining a difference value for the time-based sequence of monitoringimages by comparing the time-based sequence of reference images to thetime-based sequence of monitoring image using an image processingalgorithm, determining whether the difference value exceeds a threshold,and in response to determining the difference value exceeds thethreshold, indicating an excursion.

In another aspect, monitoring operation of a polishing system includesreceiving from a camera a time-based sequence of monitoring images of acomponent of a polishing system performing operations during polishingof a substrate, inputting the time-based sequence of monitoring imagesanalyzing into a machine learning model trained by training examples todetect excursions of the component from expected operations, andreceiving from the machine learning model an indication of an excursionof the component from expected operations. The training examplescomprise a time-based sequence of reference images of a referencecomponent of a reference polishing system performing operations during atest operation.

Certain implementations can include, but are not limited to, one or moreof the following possible advantages.

The described techniques can serve for efficient and accurateperformance analysis of components in a polishing apparatus.

First, the described techniques can enable an analysis of operations ofmultiple components in a polishing apparatus when dynamically performingrespective operations interactive to each other. In contrast toconventional image processing techniques that analyze static componentsindividually, the described techniques can analyze and detect excursionsof operations for one or more components in real-time, based on atime-based sequence of images (e.g., video frames). The describedtechniques can further enable and provide accurate analysis of componentoperations in-situ, other than analyzing each static componentseparately.

Second, sensor data acquired from in-situ monitoring systems configuredto monitor polishing of the substrate need not be used. Instead, thedescribed techniques permit an efficient overall analysis of one or morecomponents captured in video images. Additionally or in addition, thedescribed techniques can combine the analysis data with sensor data orprovide analysis of the video images as an alternative or independentcheck of components in addition to existing techniques for a moreaccurate analysis or diagnosis process.

Besides, the described techniques can generate notifications or alertsto indicate any detected excursions of one or more components in apolishing apparatus, allowing for prompt and timely human intervening orautomatic control adjustments to correct the detected excursions of oneor more components. The described techniques can eventually improveproduct quality, lower cost, and facilitate a polishing apparatus.

Also, the described techniques can store the video images that captureoperations of one or more components and allow revisiting the storedvideo images for troubling shooting or failure analysis later, leadingto a more accurate diagnosis.

Furthermore, the described techniques are easy to set up, implement, andscale up. The described techniques can be adapted to any suitablepolishing apparatus because there is no need for significantmodifications to accommodate one or more image sensors. The describedtechniques can utilize either image processing or machine learningalgorithms to analyze and detect excursions of component operations,which receive, as the solo benchmark, video images of one or morecomponents in a reference polishing apparatus performing operationsaccording to a set of reference instructions. The described techniquescan scale up for more components as long as the captured video imagescould encompass these components. Therefore, the described techniquesare ready to scale up with image sensors that can capture morecomponents with satisfactory resolutions.

The details of one or more embodiments of the invention are set forth inthe accompanying drawings and the description below. Other features,objects, and advantages are apparent from the description and drawings,and from the claims.

DESCRIPTION OF DRAWINGS

FIG. 1 is a schematic cross-sectional view of an example polishingapparatus.

FIG. 2 is a schematic cross-sectional view of an example load cup withan example carrier head.

FIG. 3 is a schematic top view of an example polishing apparatus.

FIG. 4 is a flow diagram showing an example process of excursiondetection based on video images using image processing.

FIG. 5 is a flow diagram showing an example process of excursiondetection based on video images using machine learning

Like reference numbers and designations in the various drawings indicatelike elements.

DETAILED DESCRIPTION

In an idealized process, each component of a polishing apparatusperforms operations collaboratively under a set of instructions topolish a substrate so that the substrate can have a uniform thicknessafter polishing. However, in practice, one or more components of apolishing apparatus can perform operations that deviate from therespective instructions. This can result in a non-uniform polishingprofile for a substrate undergoing polishing, collisions of one or morecomponents in the apparatus, and even failure of the apparatus. To avoidthese consequences due to operation excursions of one or morecomponents, it is of interest to monitor real-time component operationsin a polishing apparatus, detect excursions of one or more components,and even timely adjust the one or more components to operate back ontrack.

Conventionally, one or more sensors can be incorporated into a polishingapparatus to monitor one or more components in the apparatus bymeasuring one or more characteristics of an operating component or asubstrate. To name just a few examples, an optical or eddy currentin-situ monitoring system can monitor a thickness of a layer on thesubstrate during polishing, or a thermal sensor can measure atemperature of the polishing pad during polishing. Although data from asystem that monitors the substrate during polishing can provide someinformation, this may not be sufficient to detect or analyze deviationsof the system components from their expected behavior, particularly whenthere are many components in the polishing apparatus.

Moreover, some conventional techniques obtain sensor data to analyzecomponents statically—not dynamically monitor and analyze componentoperations while one or more components of a polishing apparatus areperforming operations in-situ. These conventional techniques obtainimage data of a static component and analyze the static component basedon the obtained image data. The image data can include, for example, abottom surface profile of a static retaining ring obtained through acoordinate measurement machine (CMM) for analyzing polishing edgeregions of a substrate.

The techniques described below can potentially alleviate one or more ofthe above-noted problems. A system or a polishing apparatus adopting thedescribed techniques can, using one or more video sensors (e.g., acamera), obtain a time-based sequence of reference images of a referencecomponent in a reference polishing apparatus, and capture a time-basedsequence of monitoring images of an equivalent component in anequivalent polishing apparatus. The time-based sequence of images iscaptured when one or more components are performing respectiveoperations. The system can analyze the captured image frames between thereference component and the equivalent component to determine anexcursion in real-time. In response to determine an excursion, thesystem can generate a notification, e.g., an alert indicating theexcursion in a user interface component. The system can further instructthe polishing apparatus to adjust operations of one or more componentsto correct the excursions. Optionally, the system can even terminate atleast a portion of operations executed in the polishing apparatus. Todetermine an excursion, the system can adopt different algorithmsexecuted by one or more computers in one or more locations. Thealgorithms include any suitable image processing or machine learningalgorithms.

In some implementations, the captured image frames can include one ormore reference components. The system can analyze multiple equivalentcomponents in a subset of the reference components captured in thecaptured reference image frames.

More specifically, the polishing apparatus components include a robotarm, a load cup, a conditioner arm, a transfer station, a carrier head,a slurry arm, a platen, one or more motors to drive the rotation of thecarrier head and platen. The operations of these components areinteractive. For example, the robot arm interacts with the load cup in amanner that the robot art is configured to grasp one substrate from acassette and place it horizontally (i.e., the top or bottom surface ofthe substrate is facing along with a substantially vertical position)onto a pedestal of the load cup. As another example, the carrier headinteracts with the load cup so that the carrier head is configured tograsp the substrate away from the pedestal of the load cup. The detailedstructure and operations of each component in a polishing apparatus willbe described below.

A polishing apparatus can control one or more of the components in thepolishing apparatus to perform respective operations according to a setof instructions. The set of instructions can include a plurality ofparameters predetermined by a user or automatically by the polishingapparatus to control each component's operations. The plurality ofparameters can include, for example, data specified to control aposition, or a motion of a component, or a change of physical fieldwithin the component. More specifically, the data can be an annularvelocity for a carrier head to rotate with respect to the axis ofrotation of the carrier head, or a flow rate of slurry dispensing at thenozzle of a slurry arm, to name just a few examples.

A polishing apparatus can have different sets of instructions withdifferent parameters according to different polishing requirements. Aset of instructions is also referred to as a recipe for a polishingapparatus in the description below. Once correctly executed bycomponents of a polishing apparatus, a recipe capable of causing thepolishing apparatus to polish one or more substrates substantially tofulfill a particular polishing requirement can also be referred to as a“golden recipe.” Golden recipes can be different between differentpolishing apparatuses with different components for fulling the samepolishing requirement. Ideally, a golden recipe can be adopted betweenequivalent polishing apparatuses under an identical polishingrequirement.

The term “equivalent” described above and throughout the entiredescription are used to represent a level of substantial similarity.More specifically, an equivalent polishing apparatus of a referenceapparatus can have substantially the same overall dimensions, structuraldesign, number and types of components (i.e., equivalent components),and operation pipelines as the reference apparatus. As an extremeexample, an equivalent polishing apparatus can be ideally the same copyof a reference apparatus (e.g., one of the product in the sameproduction batch), or the same model, or the same model with one or moreoptional add-ons, or trivial modifications, or have slightly differentnumbers of one or more equivalent components but still maintainsubstantially the same operations. An equivalent component of areference component can be described similarly as the equivalentpolishing apparatus. More specifically, an equivalent component can bethe same polishing component as the reference component. Alternatively,the equivalent component can be substantially identical to the referencecomponent, with optional add-ons or trivial modifications andmaintaining substantially the same operations as the referencecomponent.

The term “excursion” described above and throughout the entiredescription describes departure between the measured operations of acomponent and the operations of the reference component. The excursionis associated only with equivalent components because the referencecomponent is assumed to operate precisely according to a predeterminedrecipe. For example, the operation excursion can be quantified betweencaptured motions of an equivalent component and the correspondingreference motions of a reference component at one or more time steps. Asanother example, the operation excursion can be quantified between ameasured slurry flow rate from an equivalent nozzle and thecorresponding reference flow rate in a reference slurry nozzle. Thequantified difference can be output from different algorithms processingthe captured video images, for example, video image processing ormachine learning algorithms. The operation excursion can be determined,through various algorithms, by determining a difference between capturedimage frames of a reference component operating in a reference polishingapparatus and an equivalent component operating in an equivalentpolishing apparatus and comparing the difference against a predeterminedthreshold value. If the determined difference exceeds the predeterminedthreshold value, the system or the polishing apparatus detects anexcursion of the equivalent component.

FIG. 1 is a schematic cross-sectional view of an example polishingapparatus 20. The polishing apparatus 20 includes a rotatabledisk-shaped platen 24 on which a polishing pad 30 is situated. Theplaten 24 is operable to rotate (see arrow A in FIG. 3 ) about an axis25. For example, a motor 22 can turn a drive shaft 28 to rotate theplaten 24. The polishing pad 30 can be a two-layer polishing pad with anouter polishing layer 34 and a softer backing layer 32. The polishingapparatus 20 can include a supply port, e.g., at the end of a slurrysupply arm 39, to dispense a polishing liquid 38, such as an abrasiveslurry, onto the polishing pad 30.

Referring to FIG. 3 , the polishing apparatus 20 can include a padconditioner 90 with a conditioner disk 92 to maintain the surfaceroughness of the polishing pad 30. The conditioner disk 92 can bepositioned in a conditioner head 93 at the end of a conditioner arm 94.The arm 94 and conditioner head 93 are supported by a base 96.

The conditioner arm 94 can swing so as to sweep the conditioner head 93and conditioner disk 92 laterally across the polishing pad 30.

Referring back to FIG. 1 , the polishing apparatus 20 can also include acarrier head 70 operable to hold a substrate 10 against the polishingpad 30.

The carrier head 70 is suspended from a support structure 72, e.g., acarousel or a track, and is connected by a drive shaft 74 to a carrierhead rotation motor 76 so that the carrier head can rotate about an axis71. Optionally, the carrier head 70 can oscillate laterally, e.g., onsliders on the carousel, by movement along the track, or by therotational oscillation of the carousel itself.

The carrier head 70 can include a flexible membrane 80 having asubstrate mounting surface to contact the backside of the substrate 10,and a plurality of pressurizable chambers 82 to apply differentpressures to different zones, e.g., different radial zones, on thesubstrate 10. The carrier head 70 can include a retaining ring 84 tohold the substrate. In some implementations, the retaining ring 84 mayinclude a lower plastic portion 86 that contacts the polishing pad andan upper portion 88 of a more rigid material, e.g., a metal.

In operation, the platen 24 is rotated about its central axis 25. Thecarrier head is rotated about its central axis 71 (see arrow B in FIG. 3) and translated laterally (see arrow C in FIG. 3 ) across the topsurface of the polishing pad 30.

The polishing apparatus 20 also includes a transfer station (see FIG. 2) for loading and unloading substrates from the carrier heads 70.

The transfer station can include a plurality of load cups 8, e.g., twoload cups, adapted to facilitate the transfer of a substrate between thecarrier heads 70 and a factory interface (not illustrated) or anotherdevice (not illustrated) by a transfer robot arm (not illustrated).

The load cups 8 generally facilitate transfer between the robot arm andeach of the carrier heads 70 by loading and unloading the carrier heads70.

FIG. 2 is a schematic cross-sectional view of an example load cup 8 withan example carrier head 70. As shown in FIG. 2 , each load cup 8includes a pedestal 204 to hold the substrate 10 during aloading/unloading process. The load cup 8 also includes a housing 206that surrounds or substantially surrounds the pedestal 204.

An actuator provides relative vertical motion between the housing 206and the carrier head 70. For example, a shaft 210 can support thehousing 206 and be vertically actuatable to raise and lower the housing206. Alternatively or in addition, the carrier head 70 can movevertically. The pedestal 205 can be on-axis with the shaft 210. Thepedestal 204 can be vertically movable relative to the housing 206.

In operation, the carrier head 70 can be positioned over the load cup 8,and the housing 206 can be raised (or the carrier head 70 lowered) sothat the carrier head 70 is partially within the cavity 208. A substrate10 can begin on the pedestal 204 and be chucked onto the carrier head70, and/or begin on the carrier head 70 and be dechucked onto thepedestal 204.

The load cup 8 can further include nozzles to deliver steam for cleaningand/or pre-heating of the carrier head 70 and substrate 10. Thepolishing apparatus 20 can adjust the steam temperature, pressure, andflow rate to vary the cleaning and pre-heating of the carrier head 70and the substrate 10. In some implementations, the temperature, pressureand/or flow rate can be independently adjustable for each nozzle orbetween groups of nozzles. The flow rate of nozzles in the load cup 8can be 1-1000 cc/minute, depending on heater power and pressure.

Referring back to FIG. 1 , the polishing apparatus 20 can also include atemperature control system 100 to control the temperature of thepolishing pad 30 and/or slurry 38 on the polishing pad. The temperaturecontrol system 100 can include a cooling system 102 and/or a heatingsystem 104. At least one, and in some implementations both, of thecooling system 102 and heating system 104 operate by delivering atemperature-controlled medium, e.g., a liquid, vapor or spray, onto thepolishing surface 36 of the polishing pad 30 (or onto a polishing liquidthat is already present on the polishing pad).

The cooling system 102 can include a source 130 of liquid coolant mediumand a gas source 132 (see FIG. 3 ). The cooling system 102 or theheating system 104 can include an arm 110 that extends over the platen24 and polishing pad 30 from an edge of the polishing pad to or at leastnear (e.g., within 5% of the total radius of the polishing pad) thecenter of polishing pad 30. The arm 110 can be supported by a base 112,and the base 112 can be supported on the same frame 40 as the platen 24.The base 112 can include one or more actuators, e.g., a linear actuatorto raise or lower the arm 110, and/or a rotational actuator to swing thearm 110 laterally over the platen 24. The arm 110 is positioned to avoidcolliding with other hardware components such as the carrier heads 70,pad conditioning disk, and the slurry dispensing arm 39.

The example cooling system 102 includes multiple nozzles 120 suspendedfrom the arm 110. Each nozzle 120 is configured to spray a liquidcoolant medium, e.g., water, onto the polishing pad 30. The arm 110 canbe supported by a base 112 so that the nozzles 120 are separated fromthe polishing pad 30 by a gap 126.

FIG. 3 is a schematic top view of an example polishing apparatus 20. Asdescribed earlier in connection with FIG. 1 , the polishing apparatuscan include a heating system 102, cooling system 104, and rinse system106. As shown in FIG. 3 , the polishing apparatus can include separatearms for each of these systems. Each system can be actuated by arespective actuator. Alternatively, various subsystems can be includedin a single assembly supported by a common arm and a common actuator.

Similar to the cooling system 102, the heating system 104 is connectedto the heating medium tank with heating medium including a gas, e.g.,steam (e.g., from the steam generator 410) or heated air, or a liquid,e.g., heated water, or a combination of gas and liquid. The heatingsystem 104 can include a plurality of nozzles and an arm that extendsover the platen 24 and polishing pad 30, supported by a base 142. Thebase 142 can be supported on the same frame 40 as the platen 24. Thebase 142 can include an actuator, e.g., a linear actuator to raise orlower the arm 140, and/or a rotational actuator to swing the arm 140laterally over the platen 24. The arm is positioned to avoid collidingwith other hardware components such as the polishing head 70, padconditioning disk 92, and the slurry dispensing arm 39.

Similar to both cooling and heating systems, the high-pressure rinsesystem 106 includes a plurality of nozzles connected to a cleaning fluidtank 156 and configured to direct a cleaning fluid, e.g., water, at highintensity onto the polishing pad 30 to wash the pad 30 and remove usedslurry, polishing debris, etc.

As shown in FIG. 3 , an example rinse system 106 includes an arm thatextends over the platen 24 and is supported by a base 152, and the base152 can be supported on the same frame 40 as the platen 24. The base 152can include one or more actuators, e.g., a linear actuator to raise orlower the arm 150, and/or a rotational actuator to swing the arm 150laterally over the platen 24. The arm 150 is positioned to avoidcolliding with other hardware components such as the polishing head 70,pad conditioning disk 92, and the slurry dispensing arm 39.

In some implementations, the polishing system 20 can further include awiper blade or body 170 to distribute the polishing liquid 38 across thepolishing pad 30. Along the direction of rotation of the platen 24, thewiper blade 170 can be between the slurry delivery arm 39 and thecarrier head 70.

Referring back to FIGS. 1, 2, and 3 , the polishing apparatus 20 canalso include the controller 12 to control the operation of variouscomponents within the system. The controller 12 is also configured toreceive sensor data collected by one or more sensors measuringoperations of various components and provide feedback adjustments tochange component operations based on analysis of received sensor data.

To perform the operation of excursion detection of one or morecomponents, e.g., by comparison of data against data from referencecomponents, the system can include one or more video image sensors 14(e.g., a camera or a recorder) each positioned to having a view 16 ofone or more components of the polishing apparatus 20. Each video imagesensor 14 is configured to capture a time-based sequence of monitoringimages of operations of at least one component in a polishing apparatus.The video image sensors 14 can be positioned generally above the platen24 to have a downward perspective view of the top and/or side exteriorsurface of the various components, e.g., the carrier head 70, slurrydelivery arm 39, etc. In this position, the substrate 10 is not beingmonitored by the video image sensors 14.

The captured monitoring images can be transmitted to the controller 12in the polishing apparatus, or one or more computers external to thepolishing apparatus 20. The system can further analyze the capturedmonitoring images, based on a time-based sequence of reference imagesfor a reference component, to detect an excursion of the at least onecomponent. Optionally, the system can generate a notification upondetection of the excursion, and adjust operations of the at least onecomponent controlled by the controller 12. To adjust operations, thecontroller can transmit a feedback signal to a control mechanism (e.g.,mechanism relating to an actuator, a motor, or pressure source) foradjusting operations of the at least one component. The feedback signalcan be calculated by the controller 12 using an internal feedbackalgorithm, or received from external computers based on the capturedmonitoring images. The detail of obtaining reference images andanalyzing captured monitoring images using different algorithms will bedescribed below.

FIG. 4 is a flow diagram showing an example process 400 of the excursiondetection based on video images using image processing. The process 400can be executed by one or more computers located in one or more places.Alternatively, the process 400 can be stored as instructions in the oneor more computers. Once executed, the instructions can cause one or morecomponents of the polishing apparatus to execute the process. Forexample, the controller 12, as shown in FIGS. 1-3 , or one or morecomputers external to the polishing apparatus 20, can execute theprocess 400.

The system obtains a time-based sequence of reference images of acomponent of the polishing system performing operations during a testoperation of the polishing system (402). The system can include one ormore video cameras suitably positioned for capturing the time-basedsequence of reference images.

To obtain the time-based sequence of reference images of a referencecomponent of the polishing system, a suitable recipe is selected for thepolishing system, e.g., a golden recipe, and the associated polishingsystem is controlled, using the controller 12 to perform operationsaccording to the golden recipe. If the reference component operation issubstantially identical to the instructed operations, the video imagesobtained during the test image can be used as the time-based sequence ofreference images. For simplicity, the time-based sequence of referenceimages is also referred to as reference video in the description below.

During an actual polishing operation, e.g., as part of the fabricationprocess for an integrated circuit on a substrate, the system receivesfrom a camera a time-based sequence of monitoring images of anequivalent component of an equivalent polishing system performingoperations d (404). Similar to obtaining the reference video, the systemcan capture the time-based sequence of monitoring images using one ormore cameras or receive the time-based sequence of monitoring imagesfrom a suitable external monitoring system.

The system takes the same golden recipe for the reference component inthe polishing apparatus for the equivalent component in the equivalentpolishing apparatus, and instructs the controller 12 of the equivalentpolishing apparatus to control the equivalent component to performoperations instructed by the golden recipe. The system captures a set oftime-based monitoring images using one or more cameras. The set ofmonitoring images should encompass at least the equivalent component inthe equivalent polishing apparatus. The system can store the capturedmonitoring images for later analysis. For simplicity, the time-basedsequence of monitoring images is also referred to as monitoring video inthe description below.

The system determines a difference value for the time-based sequence ofmonitoring images by comparing the time-based sequence of referenceimages to the time-based sequence of monitoring images using an imageprocessing algorithm (406). The system adopts an image processingalgorithm to analyze the monitoring video for the equivalent component.The image processing algorithm takes as input both reference video andmonitoring video, and pre-processes both videos to decrease noises, andoptionally normalizes image pixel values for each frame of both videos.

The system can also set a respective start time for both videos for theimage processing algorithm. Starting at the respective start time, bothreference and equivalent components are in a substantially similar stateand performing substantially similar operations. For example, thereference component is a carrier head in a reference polishingapparatus. The equivalent component is a copy of the carrier head in anequivalent polishing apparatus that is also a copy of the referencepolishing apparatus. The system can set the respective start time forboth videos so that the reference carrier head and equivalent carrierhead respectively start to rotate with a respective substrate on therespective platen at the respective start time. The start time for thereference video can be different from the start time for the monitoringvideo. For example, the carrier head in the reference video starts torotate on the platen at the fifth second of the reference video.However, the equivalent carrier head in the monitoring video starts torotate on the equivalent platen at the thirty-first second of themonitoring video.

The system generates a difference value representing a similaritybetween each frame of the reference video and monitoring video startingfrom the start time. The difference value can be in any suitable form,for example, a scalar, vector, or tensor. The difference value can becalculated in any suitable manner. For example, the difference value canbe a measure of difference between each pair of pixels from each frameof the monitoring and reference videos. More specifically, the measureof difference can be a root-mean-square difference by summing squaredintensity difference pixel by pixel between each pair of frames, whichcan be the same as the output difference value. Alternatively, thedifference value can be a measure of difference between each pair ofgroup pixels or kernel outputs from each frame. The kernel receivesdifferent groups of pixels to generate features at different levels offeatures. For example, lower-level features can include lines or colors,and higher-level features can include basic shapes up to complex shapesrepresenting at least a portion of an object.

In some implementations, the difference value can represent a level ofsimilarity of a physical field obtained from both reference andmonitoring videos. The system can generate, using an image processingalgorithm, a velocity field, a pressure field, or a thermal field forcorresponding components, and compare a difference between one or morephysical fields between videos. Alternatively, the difference value canrepresent a difference of physical quantities in respective componentsbetween both reference and monitoring videos. For example, the systemcan generate average velocities, angular speed, trajectories, orvibrations of respective components in respective videos, and compare adifference between these physical quantities of components in bothvideos. The difference value can be a root-mean-squared difference for arespective physical quantity between videos, or a weighted sum ofabsolute differences for respective physical quantities between videos,to name just a few examples.

The system determines whether the difference value exceeds a threshold(408). The system can receive a pre-determined threshold from a user orautomatically generate the threshold using a particular algorithm, andassociate each obtained difference value with a respective threshold.The threshold can represent an upper bound for an absolute or relativedifference value. For example, the threshold can be 1 mm/s for adifference value in velocity, or 10% for a difference value in a thermalfield.

The system, in response to determining the difference value exceeds thethreshold, indicates an excursion (410). The system can generate analarm upon determining the difference value exceeds the threshold. Thesystem can also generate a notification on a user interface component.

Besides, the system can also generate a user presentation on the userinterface component to add information as overlays onto the capturedmonitoring video. More specifically, the system can present to a userthe monitoring video with a plurality of overlays, each of whichpresents a respective characteristic of the time-based sequence ofmonitoring images. For example, the overlay can present, for example,various physical fields (e.g., flow field, thermal field), variousphysical quantities (e.g., annular speed, trajectories), andnotifications (e.g., alarm, analysis summary).

The excursions can include various operations performed by equivalentcomponents that deviate from operations performed by the referencecomponents instructed by a reference recipe. For example, excursions canbe a departure of the component from an expected path of motion (e.g.,an expected trajectory). In these situations and in connection withFIGS. 1-3 , the component can be, for example, a carrier head 70, aplaten 24, a substrate 10 under polishing, a pedestal 204 in a load cup8, a conditioner arm 94, arms 140, 150, 110, wiper blade 170, and arobot arm.

For example, the carrier head 70 can be detected departing from anexpected rotation (e.g., rotating with a slower annular speed) bycomparing the reference and monitoring videos. As another example, thewiper blade 170 can be detected departing from an expected trajectorysuch that the wiper collides with the carrier head 70. As anotherexample, the pedestal 204 does not raise or lower as expected. Asanother example, two of the arms 94, 110, 140, and 150 collide whilesweeping. As an extreme example, the substrate 10 can be detectedescaping from the carrier head 70, shattering into pieces on the platen24.

The system can further analyze the excursion of components from anexpected path of motion to determine if one or more motors drive thecomponents in the system unexpectedly. In addition, the system candetermine, based on the excursion, if the initial calibration processhas not been conducted properly.

Furthermore, the excursion can indicate a difference of physical fieldswhen the component performs operations. In these situations and inconnection with FIGS. 1-3 , the component can be, for example, nozzlesin a load cup 8, nozzles for cooling system 102, heating system 104, andrinse system 106, a platen 24, and slurry dispensing arm 39.

For example, the slurry flow rate dispensed from the dispensing arm 39can be detected as having a slower flow rate than the reference video.As another example, the thermal field on the platen 24 can have one ormore regions with high temperatures than the reference video. As anotherexample, the nozzles in the load cup 8 can be detected as having a highwater pressure that would cause an overspray. The overspray can also bedetected in other components of a polishing apparatus, for example, anoverspray in the slurry dispensing arm 39. As an extreme example, theplaten 24 is detected overheated, and the nozzles of cooling system 102are clogged.

After determining the difference value exceeds the threshold, the systemcan generate corrective operations to adjust the equivalent componentperforming operations to decrease the difference value. The system cangenerated instructions for the adjusted operations on one or morecomputers external to the polishing apparatus 20, or the system caninstruct the controller 12 to generate the instructions. Then, thecontroller 12 can control corresponding components based on theinstructions to adjust respective operations.

The system can scale up the analysis process readily to simultaneouslydetect excursions of a plurality of components in a polishing system.

The system first obtains a respective time-based sequence of referencesimages that encompass a plurality of components of the polishing systemperforming operations during a respective test operation of thepolishing system.

The system then receives from a camera a time-based sequence ofmonitoring images that encompass one or more equivalent components of anequivalent polishing system performing operations during polishing of asubstrate. The plurality of components captured in the reference videoshould include all types of the one or more equivalent components in theequivalent polishing apparatus. The system can perform similar steps of406, 408, and 410 for each of the one or more equivalent components.

FIG. 5 is a flow diagram showing an example process 500 of excursiondetection based on video images using machine learning. The process 500can be executed by one or more computers located in one or more places.Alternatively, the process 500 can be stored as instructions in the oneor more computers. Once executed, the instructions can cause one or morecomponents of the polishing apparatus to execute the process. Forexample, the controller 12, as shown in FIGS. 1-3 , or one or morecomputers external to the polishing apparatus 20, can execute theprocess 500.

The system receives from a camera a time-based sequence of monitoringimages of a component of a polishing system performing operations duringpolishing of a test substrate (502). Similar to process 400 as describedabove, the system can capture the time-based sequence of monitoringimages using one or more cameras or receive the time-based sequence ofmonitoring images from a suitable external monitoring system. The systemcaptures the monitoring video of the component performing operationsinstructed by a reference recipe.

The system inputs the time-based sequence of monitoring images analyzinginto a machine learning model trained by training examples to detectexcursions of the component from expected operations; wherein thetraining examples comprise a time-based sequence of reference images ofa reference component of a reference polishing system performingoperations during a test operation (504) and a classification of thesequence of reference images, e.g., as normal operation or an excursion.The classification can also identify the type of excursion, e.g.,failure of the pedestal in the load cup to lift, deviation of thecarrier head from the expected sweep position, etc.

The system can obtain data representing a plurality of training examplesfrom external memory or collect training examples using one or morecameras. Each training example is a time-based sequence of images of acomponent performing operations. To collect training examples, thesystem can instruct a plurality of equivalent polishing apparatuses withequivalent components to perform operations using a reference recipe(i.e., the same recipe for capturing monitoring videos) and capturerespective monitoring videos for each equivalent component. The systemcan train the neural network based on the captured respective monitoringvideos.

In some implementations, the system can obtain the training examplesusing both a reference component in a reference polishing apparatus andone or more equivalent components in respective equivalent polishingapparatuses. Optionally, the system can include weight values for eachtraining example during training the neural network, and set a greaterweight value for the training sample with the reference video.

After training the neural network, the system can perform inferencecomputations using input data (e.g., monitoring videos) of equivalentcomponents. The equivalent components are substantially similarcomponents to those in the training samples.

The system receives from the machine learning model an indication of anexcursion of the component from expected operations (506). Similar tostep 410 of process 400, the system can detect an excursion of anequivalent component based on the input data (e.g., a monitoring videoof the component). The excursions are similarly described in connectionwith process 400. The system can similarly generate correctiveoperations to adjust the equivalent component performing operations.

The system can store the trained neural network on one or more computersin one or more locations. One or more processors can simultaneouslyaccess the stored neural network to accelerate inference operations(e.g., parallel calculations). The system can continue training theneural network with newly captured training examples. Similarly, thesystem can be scaled up to monitor operations' excursion for a pluralityof components in a polishing apparatus at the same time.

As used in the instant specification, the term substrate can include,for example, a product substrate (e.g., which includes multiple memoryor processor dies), a test substrate, a bare substrate, and a gatingsubstrate. The substrate can be at various stages of integrated circuitfabrication, e.g., the substrate can be a bare wafer, or it can includeone or more deposited and/or patterned layers. The term substrate caninclude circular disks and rectangular sheets.

The above described polishing apparatus and methods can be applied in avariety of polishing systems. Either the polishing pad, or the carrierheads, or both can move to provide relative motion between the polishingsurface and the substrate. For example, the platen may orbit rather thanrotate. The polishing pad can be a circular (or some other shape) padsecured to the platen. Some aspects of the endpoint detection system maybe applicable to linear polishing systems, e.g., where the polishing padis a continuous or a reel-to-reel belt that moves linearly. Thepolishing layer can be a standard (for example, polyurethane with orwithout fillers) polishing material, a soft material, or afixed-abrasive material. Terms of relative positioning are used; itshould be understood that the polishing surface and substrate can beheld in a vertical orientation or some other orientation.

Control of the various systems and processes described in thisspecification, or portions of them, can be implemented in a computerprogram product that includes instructions that are stored on one ormore non-transitory computer-readable storage media, and that areexecutable on one or more processing devices. The systems described inthis specification, or portions of them, can be implemented as anapparatus, method, or electronic system that may include one or moreprocessing devices and memory to store executable instructions toperform the operations described in this specification.

Embodiments of the classification and training of a machine learningmodel described in this specification can be implemented in digitalelectronic circuitry, in tangibly-embodied computer software orfirmware, in computer hardware, including the structures disclosed inthis specification and their structural equivalents, or in combinationsof one or more of them. Embodiments of the subject matter described inthis specification can be implemented as one or more computer programs,i.e., one or more modules of computer program instructions encoded on atangible non transitory storage medium for execution by, or to controlthe operation of, data processing apparatus. The computer storage mediumcan be a machine-readable storage device, a machine-readable storagesubstrate, a random or serial access memory device, or a combination ofone or more of them. Alternatively or in addition, the programinstructions can be encoded on an artificially generated propagatedsignal, e.g., a machine-generated electrical, optical, orelectromagnetic signal, that is generated to encode information fortransmission to suitable receiver apparatus for execution by a dataprocessing apparatus.

A computer program, which may also be referred to or described as aprogram, software, a software application, an app, a module, a softwaremodule, a script, or code, can be written in any form of programminglanguage, including compiled or interpreted languages, or declarative orprocedural languages; and it can be deployed in any form, including as astandalone program or as a module, component, subroutine, or other unitsuitable for use in a computing environment. A program may, but neednot, correspond to a file in a file system. A program can be stored in aportion of a file that holds other programs or data, e.g., one or morescripts stored in a markup language document, in a single file dedicatedto the program in question, or in multiple coordinated files, e.g.,files that store one or more modules, sub programs, or portions of code.A computer program can be deployed to be executed on one computer or onmultiple computers that are located at one site or distributed acrossmultiple sites and interconnected by a data communication network.

The processes and logic flows described in this specification can beperformed by one or more programmable computers executing one or morecomputer programs to perform functions by operating on input data andgenerating output. The processes and logic flows can also be performedby special purpose logic circuitry, e.g., an FPGA or an ASIC, or by acombination of special purpose logic circuitry and one or moreprogrammed computers.

Computers suitable for the execution of a computer program can be basedon general or special purpose microprocessors or both, or any other kindof central processing unit. Generally, a central processing unit willreceive instructions and data from a read only memory or a random accessmemory or both. The essential elements of a computer are a centralprocessing unit for performing or executing instructions and one or morememory devices for storing instructions and data. The central processingunit and the memory can be supplemented by, or incorporated in, specialpurpose logic circuitry. Generally, a computer will also include, or beoperatively coupled to receive data from or transfer data to, or both,one or more mass storage devices for storing data, e.g., magnetic,magneto optical disks, or optical disks. However, a computer need nothave such devices.

Computer readable media suitable for storing computer programinstructions and data include all forms of non-volatile memory, mediaand memory devices, including by way of example semiconductor memorydevices, e.g., EPROM, EEPROM, and flash memory devices; magnetic disks,e.g., internal hard disks or removable disks; magneto optical disks; andCD ROM and DVD-ROM disks.

Data processing apparatus for implementing machine learning models canalso include, for example, special-purpose hardware accelerator unitsfor processing common and compute-intensive parts of machine learningtraining or production, i.e., inference, workloads.

Machine learning models can be implemented and deployed using a machinelearning framework, e.g., a TensorFlow framework, a Microsoft CognitiveToolkit framework, an Apache Singa framework, or an Apache MXNetframework.

Embodiments of the subject matter described in this specification can beimplemented in a computing system that includes a back end component,e.g., as a data server, or that includes a middleware component, e.g.,an application server, or that includes a front end component, e.g., aclient computer having a graphical user interface, a web browser, or anapp through which a user can interact with an implementation of thesubject matter described in this specification, or any combination ofone or more such back end, middleware, or front end components. Thecomponents of the system can be interconnected by any form or medium ofdigital data communication, e.g., a communication network. Examples ofcommunication networks include a local area network (LAN) and a widearea network (WAN), e.g., the Internet.

The computing system can include clients and servers. A client andserver are generally remote from each other and typically interactthrough a communication network. The relationship of client and serverarises by virtue of computer programs running on the respectivecomputers and having a client-server relationship to each other. In someembodiments, a server transmits data, e.g., an HTML page, to a userdevice, e.g., for purposes of displaying data to and receiving userinput from a user interacting with the device, which acts as a client.Data generated at the user device, e.g., a result of the userinteraction, can be received at the server from the device.

While this specification contains many specific implementation details,these should not be construed as limitations on the scope of anyinvention or on the scope of what may be claimed, but rather asdescriptions of features that may be specific to particular embodimentsof particular inventions. Certain features that are described in thisspecification in the context of separate embodiments can also beimplemented in combination in a single embodiment. Conversely, variousfeatures that are described in the context of a single embodiment canalso be implemented in multiple embodiments separately or in anysuitable subcombination. Moreover, although features may be describedabove as acting in certain combinations and even initially claimed assuch, one or more features from a claimed combination can in some casesbe excised from the combination, and the claimed combination may bedirected to a subcombination or variation of a sub combination.

Particular embodiments of the subject matter have been described. Otherembodiments are within the scope of the following claims.

Other embodiments are within the scope of the following claims.

What is claimed is:
 1. A polishing system, comprising: a platen tosupport a polishing pad; a carrier head to hold a substrate against thepolishing pad; a component selected from the group including the platen,the carrier head, a conditioner arm, a load cup, or a robot arm; acamera positioned to capture a time-based sequence of monitoring imagesof the component; and a controller configured to cause the component toperform operations in accord with a reference recipe, receive from thecamera a time-based sequence of training images of the componentperforming the operations, and train a machine learning model using thetraining images to detect and indicate excursions of the component fromexpected operations.
 2. The system of claim 1, wherein the machinelearning model is configured to, using an object tracking algorithm,determine motions of the component based on the input time-basedsequence of monitoring images.
 3. The system of claim 1, wherein themachine learning model is configured to predict a physical field profileof the component based on the input time-based sequence of monitoringimages.
 4. The system of claim 3, wherein the physical field profile ofthe component comprises: a temperature field of a rotating polishingpad, or a pressure field applied on a wafer.
 5. The system of claim 3,wherein the excursion comprises departure of the component from anexpected path of motion.
 6. The system of claim 1, wherein thecontroller is configured to generate an alarm in response to anindication of an excursion.
 7. A computer program product, tangiblyembodied in computer-readable media, comprising instructions to causeone or more computers to: receive from a camera a time-based sequence oftraining images of a component of a polishing system performingoperations in accord with a reference recipe; and train a machinelearning model using the training images to detect and indicateexcursions of the component from expected operations.
 8. The computerprogram product of claim 7, wherein the machine learning model isconfigured to, using an object tracking algorithm, determine motions ofthe component based on the input time-based sequence of monitoringimages.
 9. The computer program product of claim 7, wherein the machinelearning model is configured to predict a physical field profile of thecomponent based on the input time-based sequence of monitoring images.10. The computer program product of claim 9, wherein the physical fieldprofile of the component comprises: a temperature field of a rotatingpolishing pad, or a pressure field applied on a wafer.
 11. The computerprogram product of claim 7, wherein the excursion comprises departure ofthe component from an expected path of motion.
 12. The computer programproduct of claim 7, wherein the excursion comprises an overspray offluid from a fluid dispenser.
 13. A method for making a monitoringsystem for a polishing system, the method comprising: cause a componentof a polishing system to perform operations in accord with a referencerecipe, monitor the component with a camera to generate a time-basedsequence of training images of the component; and train a machinelearning model using the training images to detect and indicateexcursions of the component from expected operations.
 14. The method ofclaim 13, wherein the excursion comprises departure of the componentfrom an expected path of motion.
 15. The method of claim 14, wherein thecomponent comprises one of a carrier head, a conditioner arm, a loadcup, a platen or a robot arm.